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A method of determining the extent of blocking artifacts in a digital image 

(57) A digital image processing method determines 
the extent of blocking artifacts in a digital image by first 
forming a column difference image and averaging the 
values in the columns in the column difference image to 
produce a column difference array. The average of the 
values in the column difference array that are separated 
by one block width are computed to produce a block av- 
eraged column difference array. Then, the peak value 
in the block averaged column difference array is located, 
and the mean value of the block averaged column dif- 
ference array (excluding the peak value) is calculated 
to produce a column base value, and the ratio between 
the peak value and the base value are computed to pro- 
duce a column ratio. The foregoing steps are repeated 
in the row direction to produce a row ratio. Finally, the 
column and row ratios are employed as a measure of 
the extent of blocking artifacts in the digital image. 
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Description 

FIELD OF THE INVENTION 

[0001] This invention relates to the field of digital im- 5 
age processing, and more particularly to methods of de- 
termining the extent of blocking artifacts in a digital im- 
age. 

BACKGROUND OF THE INVENTION 10 

[0002] Digital images contain enormous amounts of 
data. Storage of such data on digital media is generally 
expensive, and the transmission of digital images re- 
quires either a large bandwidth or a long period of time. 15 
Many algorithms have been developed to compress im- 
age data by removing visually redundant information 
from the image. Discrete-cosine-transform-based 
(DCT-based) compression has been the most popular 
among existing techniques. The DCT has very good en- 20 
ergy compaction and data decorrelation properties. 
Moreover, the DCT can be computed using fast algo- 
rithms and efficiently implemented using very large 
scale integration (VLSI) techniques. In DCT-based cod- 
ing, an image is partitioned into small square blocks 25 
(typically 8x8) and the DCT is computed over these 
blocks to remove the local spatial correlation. In order 
to achieve high compression, quantization of the DCT 
coefficients is then performed. Quantization is an irre- 
versible process that causes loss of information and dis- 30 
tortions in the decompressed image. After quantization, 
the redundancy in the data is further reduced using en- 
tropy coding. At the decoder end, the received data is 
decoded, dequantized, and reconstructed by the in- 
verse DCT In general, a typical 8-bit gray-level image 35 
can be coded with a compression ratio of up to 1 0: 1 with- 
out noticeable artifacts. 

[0003] However, at low bit rates the reconstructed im- 
ages generally suffer from visually annoying artifacts as 
a result of very coarse quantization. One major artifact 40 
is the blocking effect, which appears as artificial block 
boundaries between adjacent blocks. At a low bit rate, 
each block is represented mainly by the first few low- 
frequency coefficients and, since each block is proc- 
essed independently, no interblock correlation is ac- 45 
counted for in standard block DCT-based coding 
schemes. Therefore, discontinuity across the block 
boundary becomes noticeable. 

[0004] There are many techniques developed to re- 
duce the blocking effect, (e.g., H. C. Reeve HI and J. S. 50 
Lim, "Reduction of blocking effect in image coding," 
ICASSP, pp. 1212-1215, 1983). Since most of these 
techniques employ some kind of image filtering tech- 
nique and thus reduce image sharpness to some extent, 
it is imperative that these techniques not be used on a 55 
"good image," i.e., one that has not been compressed 
highly enough to exhibit the blocking artifacts. There- 
fore, it is necessary to develop a triage algorithm that 



measures the extent of blocking artifacts in a digital im- 
age. 

[0005] Certain blocking artifact removal algorithms re- 
quire prior knowledge of block boundary locations. How- 
ever, since digital images compressed with a DCT- 
based technique can be further modified, for example, 
through cropping and zooming, the block boundary lo- 
cations may have changed and become unknown at the 
time of blocking artifact removal. Thus, for those block- 
ing artifact removal algorithms, there is a need for an 
automatic technique for detecting block boundary loca- 
tions in digital images having blocking artifacts. 
[0006] Because digital images may or may not have 
been compressed, and may have gone through different 
degrees of compression, the extent of the blocking arti- 
facts varies from nonexistent to visually objectionable. 
Therefore, There is a need for determining the extent of 
the blocking artifacts in order to decide whether to per- 
form artifact removing procedures. Furthermore, be- 
cause virtually all artifact removal algorithms remove 
blocking artifacts at the expense of image detail, there 
is a need for controlling the optimal amount of filtering 
in order to strive for a good trade-off between artifact 
removal and image detail preservation. 
[0007] A conventional method for determining the ex- 
tent of the blocking artifacts involves taking the ratio be- 
tween the size of the compressed JPEG file and the size 
of the image, which corresponds to the amount of un- 
compressed image data. This ratio is commonly re- 
ferred to as the compression ratio. However, the com- 
pression ratio is not necessarily a good measure of the 
quality of the compressed image, nor the extent of the 
blocking artifacts. In general, at the same compression 
ratio, a busy image would look worse than a less busy 
image because busy images are harder to compress 
(therefore a busy image has lost more details than the 
less busy image). Therefore, there is a need to define a 
more perceptually accurate measure of the blocking ar- 
tifacts. 

SUMMARY OF THE INVENTION 

[0008] The present invention is directed to overcom- 
ing one or more of the problems set forth above. Briefly 
summarized, according to one aspect of the present in- 
vention, a digital image processing method for determin- 
ing the extent of blocking artifacts in a digital image, in- 
cludes the steps of: forming a column difference image; 
averaging the values in the columns in the column dif- 
ference image to produce a column difference array; 
computing the average of the values in the column dif- 
ference array that are separated by one block width to 
produce a block averaged column difference array; lo- 
cating the peak value in the block averaged column dif- 
ference array; calculating the mean value of the block 
averaged column difference array excluding the peak 
value to produce a column base value; computing the 
ratio between the peak value and the base value to pro- 
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duce a column ratio; repeating the above-mentioned 
steps in the row direction to produce a row ratio; and 
employing the column and row ratios as a measure of 
the extent of blocking artifacts in the digital image. 

5 

ADVANTAGES 

[0009] The present invention has the advantage that 
it measures the extent of blocking artifacts in a digital 
image based on image content. The present invention 10 
also has the advantage that it automatically detects 
block boundary locations even if a digital image has 
been modified by cropping and zooming. 

BRIEF DESCRIPTION OF THE DRAWINGS 15 



image. The digital image may be a grayscale image con- 
taining the intensity component, or a color image con- 
taining RGB components. For color images, a color 
transform is typically performed before image compres- 
sion to take advantage of the redundancy in the color 
perception of the human visual system (HVS). For ex- 
ample, in JPEG compression, a color image is first con- 
verted from RGB to YCbCr color space with the follow- 
ing equations: 

Y=1 6+65.481 * R+1 28.553* G+24.966* B; 



Cb=128-37.797*R-74.203*G+112*B; 



10 



[0010] 

FIG. 1 is a block diagram of an embodiment of cur- 
rent invention. 

FIG.2 shows a plot of a typical block averaged dif- 
ference array. 

DETAILED DESCRIPTION OF THE INVENTION 

[0011] In the following description, a preferred em- 
bodiment of the present invention will be described as 
an image processing method that would ordinarily be 
embodied as a software program. Those skilled in the 
art will readily recognize that the equivalent of such soft- 
ware may also be constructed in hardware. Because im- 
age manipulation algorithms and systems for running 
such algorithms are well known, the present description 
will be directed in particular to algorithms and systems 
forming part of, or cooperating more directly with, the 
method in accordance with the present invention. Other 
aspects of such algorithms and systems, and hardware 
and/or software for producing and otherwise processing 
the image signals involved therewith, not specifically 
shown or described herein may be selected from such 
systems, algorithms, components and elements known 
in the art. Given the method as described according to 
the invention in the following materials, software not 
specifically shown, described or suggested herein that 
is useful for implementation of the invention is conven- 
tional and within the ordinary skill in such arts. 
[0012] Furthermore, as used herein, the computer 
program may be stored in a computer readable storage 
medium, which may comprise, for example; magnetic 
storage media such as a magnetic disk (such as a floppy 
disk or a hard drive) or magnetic tape; optical storage 
media such as an optical disc, optical tape, or machine 
readable bar code; solid state electronic storage devic- 
es such as random access memory (RAM), or read only 
memory (ROM); or any other physical device or medium 
employed to store a computer program. 
[0013] The present invention relates to a method for 
determining the extent of blocking artifacts in a digital 



Cr=128+112*R-93.786*G-18.214*B; 

20 where Y is the intensity component, and Cb and Cr are 
the two chrominance components. R, G and B are all 
normalized to 1. 

[0014] Referring to Figure 1 , for each image compo- 
nent, a column difference image is first generated in step 

25 10 by calculating the absolute difference between two 
adjacent columns. For example, the column difference 
image is generated by subtracting the pixel values of the 
second column of the image from the pixel values of the 
first column of the image to generate a column of differ- 

30 ence values, and then setting the pixel values of the first 
column of the column difference image as the absolute 
values of the column of difference values. The same 
procedure is repeated to set the rest of the columns of 
the column difference image except for the last column, 

35 where all the values of that column are set to zero. 
[0015] The column difference image is further aver- 
aged in the vertical direction to generate a one-dimen- 
sional column difference array VA in a step 12. Assum- 
ing that the original image has M rows and N columns 

40 of image data, the column difference array VA should 
have N entries. To keep image edges from contributing 
to the detection, the contribution of a pixel in the column 
difference image is discarded if the magnitude of the in- 
tensity gradient of the corresponding pixel in the original 

45 image component is larger than a threshold value T. So- 
bel operators are used as the intensity gradient opera- 
tors to generate an intensity gradient image with the in- 
tensity gradient equal to the sum of the absolute values 
from the horizontal and vertical Sobel operators. In the 

50 current embodiment of the invention, the standard de- 
viation (SD) of the intensity gradient image is calculated, 
and the threshold value T is set as twice the standard 
deviation SD. 

[0016] The column difference array VA is further av- 
55 eraged using a periodicity of the JPEG block width (8) 
in a step 14 to generate a block averaged column dif- 
ference array VAA, which has eight entries. In other 
words, every eighth entry of the column difference array 
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VA will be averaged, and the result will be used to set 
the eight entries of the block averaged column differ- 
ence array VAA. 

[001 7] From the block averaged column difference ar- 
ray VAA, the maximum value is first located in a step 1 6 5 
and defined as the column peak value, then the mean 
value of the block averaged column difference array 
VAA is calculated in a step 18 excluding the peak value. 
The mean value is defined as the column base value. 
Finally, the ratio between the column peak value and the 10 
column base value is calculated in a step 20 to generate 
a column ratio. 

[001 8] Figure 2 shows a typical block averaged differ- 
ence array with 8 elements (index starts at 0) from a 
typical digital image. The x-axis is the array index and 15 
the y-axis is magnitude. In this case, the peak value 
(2.62) occurs at index 7, the mean value is 1 .03, and the 
ratio is about 2.6. 

[0019] Referring to Figure 1 , the row peak value, row 
base value and row ratio are also calculated analogous- 20 
Jy in the series of steps 10' - 20', where the common 
reference numerals indicate that the respective opera- 
tions performed on the rows are analogous to the oper- 
ations performed on the columns. More specifically, a 
row difference array HA and a block averaged row dif- 25 
ference array HAA are generated in the block 12' and 
14', respectively, and similar measures based on these 
arrays are developed in the blocks 16* - 20'. 
[0020] The column and row ratios are good indicators 
of the presence of the blocking artifacts in the column 30 
and row directions, respectively. Therefore, a preferred 
measure (calculated in a step 22) for the extent of block- 
ing artifacts is the average values of the column and row 
ratios for images with statistically equally distributed 
horizontal and vertical structures. A more aggressive 35 
measure is the minimum value of the column and row 
ratios. If the measure exceeds a predetermined thresh- 
old, a conventional artifact-removing algorithm is trig- 
gered, such as the algorithm described in the aforemen- 
tioned Reeve and Lim article. Based on a study of hu- 40 
man subject responses, a preferred threshold value is 
determined as 2.0. 

[0021] The visibility or objectionability of the blocking 
artifacts may depend on the actual structure in an im- 
age. For example, textured areas tend to hide the block- 45 
ing artifacts better than flat areas. The column and row 
base values are good indicators of the amount of tex- 
tures. To achieve a higher degree of adaptivity to the 
image content, the measure of the extent of the blocking 
artifacts can be defined as a function of both the column/ 50 
row ratios and the column/row base values. In general, 
the higher the base values, meaning a higher degree of 
texture activity (capable of hiding a larger extent of 
blocking artifacts), a higher threshold on the artifact 
measure should be used. In practice, a look-up-table 55 
can be built to quantitatively characterize the relation- 
ship between the base values and the threshold. 
[0022] If the measure for the extent of blocking arti- 



facts exceeds a predetermined threshold, the artifact- 
removing algorithm is triggered. The indexes of peak 
value in the block averaged column difference array 
VAA and the block averaged row difference array HAA 
are the JPEG block boundary locations in the horizontal 
and vertical directions for the image component. For ex- 
ample, if the index for the peak value in the block aver- 
aged column difference array VAA is 4, then it is as- 
sumed that JPEG block boundary will appear at column 
no. 4, 12, in the image component. Note that the in- 
dexes for the columns of the image component as well 
as for the block averaged column difference array VAA 
and the block averaged row difference array HAA start 
at zero in this embodiment. 

[0023] In the case where digital images are not only 
cropped but also zoomed (shrunk or magnified), the 
block size is no longer the original JPEG block size of 
8. For potentially magnified digital images, the above- 
mentioned process for determining the extent of the 
JPEG blocking artifacts can be repeated for a predeter- 
mined series of hypothetical block width values to pro- 
duce a series of ratios between the peak value and the 
base value. The hypothetical block width that produces 
the maximum artifact measure is chosen to be the ef- 
fective block size after zooming. The zooming factor is 
in turn determined to be the ratio between the effective 
block size and the original block size (i.e., 8). 



Claims 

1 . A method of determining the extent of blocking ar- 
tifacts in a digital image formed of rows and columns 
of pixels, comprising the steps of: 

a) forming a column difference image from the 
difference between columns in the digital im- 
age; 

b) averaging the values in the columns in the 
column difference image to produce a column 
difference array; 

c) computing the average of the values in the 
column difference array that are separated by 
one block width to produce a block averaged 
column difference array; 

d) locating a peak value in the block averaged 
column difference array; 

e) calculating a mean value of the block aver- 
aged column difference array excluding the 
peak value to produce a column base value; 

f) computing the ratio between the peak value 
and the column base value to produce a column 
ratio; 

g) repeating steps a) - f) in the row direction to 
produce a row ratio; and; 

h) employing the column and row ratios as a 
measure of the extent of blocking artifacts in the 
digital image. 
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2. The method as claimed in claim 1 , further compris- 
ing the steps of: 

a) generating an intensity gradient image from 
the digital image; 

b) calculating the standard deviation of the gra- 
dient image; 

c) setting a threshold value that is twice the 
standard deviation; and 

d) in computing the average of the values in the 
column and row difference arrays, excluding 
pixels whose gradients are larger than the 
threshold value. 

3. The method as claimed in claim 1, wherein the 
measure of the extent of blocking artifacts is the av- 
erage of the column and row ratios. 

4. The method as claimed in claim 1, wherein the 
measure of the extent of blocking artifacts is a func- 
tion of the column and row ratios and the column 
and row base values. 



has been applied, comprising the steps of: 

a) computing the average of the values in the 
column difference array that are separated by 
a hypothetical block width to produce a block 
averaged column difference array; 

b) repeating steps d) - h) in claim 1 for a prede- 
termined series of hypothetical block width val- 
ues to produce a series of ratios between the 
peak value and the base value; 

c) selecting the block width that corresponds to 
the peak ratio as an effective block width after 
post-compression image magnification; and 

d) determining a zooming factor to be the ratio 
between the effective block size and the origi- 
nal block size. 



5. The method as claimed in claim 1, wherein the 
measure of the extent of blocking artifacts is a look- 25 
up-table of the column and row ratios and the col- 
umn and row base values. 



6. The method as claimed in claim 1 , further compris- 
ing the steps of: 30 

a) if the measure of the extent of blocking arti- 
facts exceeds a predetermined threshold, de- 
termining the location of the row and column 
block boundaries as the locations of the column 35 
and row peak values; and 

b) applying a blocking artifact removal algo- 
rithm to the image according to the location and 
extent of the blocking artifacts. 

40 

7. The method as claimed in claim 1 , for potentially 
magnified digital images, further comprising the 
steps of: 



a) computing the average of the values in the 45 
column difference array that are separated by 

a hypothetical block width to produce a block 
averaged column difference array; 

b) repeating steps d) - h) in claim 1 for a prede- 
termined series of hypothetical block width val- so 
ues to produce a series of ratios between the 
peak value and the base value; and 

c) selecting the block width that corresponds to 
the peak ratio as an effective block width after 
post-compression image magnification. 55 

8. A method as claimed in claim 1 , for determining the 
zooming factor to which a compressed digital image 
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